35 research outputs found
Knowledge base question answering with a matching-aggregation model and question-specific contextual relations
National Research Foundation (NRF) Singapore under International Research Centres in Singapore Funding Initiativ
Multi-level head-wise match and aggregation in transformer for textual sequence matching
Transformer has been successfully applied to many natural language processing
tasks. However, for textual sequence matching, simple matching between the
representation of a pair of sequences might bring in unnecessary noise. In this
paper, we propose a new approach to sequence pair matching with Transformer, by
learning head-wise matching representations on multiple levels. Experiments
show that our proposed approach can achieve new state-of-the-art performance on
multiple tasks that rely only on pre-computed sequence-vector-representation,
such as SNLI, MNLI-match, MNLI-mismatch, QQP, and SQuAD-binary.Comment: AAAI 2020, 8 page
R Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context
With the help of Chain-of-Thought (CoT) prompting, Large Language Models
(LLMs) have achieved remarkable performance on various reasoning tasks.
However, most of them have been evaluated under noise-free context and the
dilemma for LLMs to produce inaccurate results under the noisy context has not
been fully investigated. Existing studies utilize trigger sentences to
encourage LLMs to concentrate on the relevant information but the trigger has
limited effect on final answer prediction. Inspired by interactive CoT method,
where intermediate reasoning steps are promoted by multiple rounds of
interaction between users and LLMs, we propose a novel prompting method, namely
R prompting, for CoT reasoning under noisy context. Specifically, R
prompting interacts with LLMs to perform key sentence extraction, variable
declaration and answer prediction, which corresponds to a thought process of
reviewing, rephrasing and resolving. The responses generated at the last
interaction will perform as hints to guide toward the responses of the next
interaction. Our experiments show that R prompting significantly
outperforms existing CoT prompting methods on five reasoning tasks under noisy
context. With GPT-3.5-turbo, we observe 3.7% accuracy improvement on average on
the reasoning tasks under noisy context compared to the most competitive
prompting baseline. More analyses and ablation studies show the robustness and
generalization of R prompting method in solving reasoning tasks in LLMs
under noisy context
Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation
The task of Question Generation over Knowledge Bases (KBQG) aims to convert a
logical form into a natural language question. For the sake of expensive cost
of large-scale question annotation, the methods of KBQG under low-resource
scenarios urgently need to be developed. However, current methods heavily rely
on annotated data for fine-tuning, which is not well-suited for few-shot
question generation. The emergence of Large Language Models (LLMs) has shown
their impressive generalization ability in few-shot tasks. Inspired by
Chain-of-Thought (CoT) prompting, which is an in-context learning strategy for
reasoning, we formulate KBQG task as a reasoning problem, where the generation
of a complete question is splitted into a series of sub-question generation.
Our proposed prompting method KQG-CoT first retrieves supportive logical forms
from the unlabeled data pool taking account of the characteristics of the
logical form. Then, we write a prompt to explicit the reasoning chain of
generating complicated questions based on the selected demonstrations. To
further ensure prompt quality, we extend KQG-CoT into KQG-CoT+ via sorting the
logical forms by their complexity. We conduct extensive experiments over three
public KBQG datasets. The results demonstrate that our prompting method
consistently outperforms other prompting baselines on the evaluated datasets.
Remarkably, our KQG-CoT+ method could surpass existing few-shot SoTA results of
the PathQuestions dataset by 18.25, 10.72, and 10.18 absolute points on BLEU-4,
METEOR, and ROUGE-L, respectively.Comment: Accepted by EMNLP 2023 main conferenc
Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Prompts
Zero-shot Visual Question Answering (VQA) is a prominent vision-language task
that examines both the visual and textual understanding capability of systems
in the absence of training data. Recently, by converting the images into
captions, information across multi-modalities is bridged and Large Language
Models (LLMs) can apply their strong zero-shot generalization capability to
unseen questions. To design ideal prompts for solving VQA via LLMs, several
studies have explored different strategies to select or generate
question-answer pairs as the exemplar prompts, which guide LLMs to answer the
current questions effectively. However, they totally ignore the role of
question prompts. The original questions in VQA tasks usually encounter
ellipses and ambiguity which require intermediate reasoning. To this end, we
present Reasoning Question Prompts for VQA tasks, which can further activate
the potential of LLMs in zero-shot scenarios. Specifically, for each question,
we first generate self-contained questions as reasoning question prompts via an
unsupervised question edition module considering sentence fluency, semantic
integrity and syntactic invariance. Each reasoning question prompt clearly
indicates the intent of the original question. This results in a set of
candidate answers. Then, the candidate answers associated with their confidence
scores acting as answer heuristics are fed into LLMs and produce the final
answer. We evaluate reasoning question prompts on three VQA challenges,
experimental results demonstrate that they can significantly improve the
results of LLMs on zero-shot setting and outperform existing state-of-the-art
zero-shot methods on three out of four data sets. Our source code is publicly
released at \url{https://github.com/ECNU-DASE-NLP/RQP}